For example, in a broadcasting station, a video camera having an image pickup device that picks up an image of a real world via an optical system picks up an image of light from an object such as scenery or the like in the real world via the optical system and samples the light in a spatiotemporal direction, whereby image data is obtained.
Specifically, in the video camera, pixels of minute size of the image pickup device receive light of the real world, whereby the light is sampled in a spatial direction. Further, in the pixels of the image pickup device, the light of the real world is sampled in a temporal direction by being received for a predetermined time and being subjected to photoelectric conversion. The image pickup device outputs image data composed of pixel values according to amounts of charge corresponding to the light sampled in the spatial direction and the temporal direction as described above.
Further, in the broadcasting station, using the image data obtained by the video camera as a material, the image data as material is edited, subjected to signal processing such as MPEG (Moving Picture Experts Group) coding and the like, and transmitted.
When the image data is MPEG-coded as described above, a receiving side performs MPEG decoding. Because the MPEG coding subjects the image data to a DCT (Discrete Cosine Transform) in block units of a predetermined size, and further quantizes the image data, distortion such as block distortion or the like occurs in a decoded image obtained as a result of the MPEG decoding.
Accordingly, the present applicant has previously proposed a method of remedying distortion such as block distortion or the like occurring in a decoded image obtained as a result of the MPEG decoding by a technique referred to as class classification adaptive processing (see Patent Document 1, for example).
Incidentally, for example, even when a monitor on the receiving side is a high-performance monitor capable of displaying an image that faithfully reproduces the real world, image quality of an image displayable on the monitor is limited by the performance of the video camera of the broadcasting station.
In addition, in the real world, a user can have a bodily sensation of all of scenery seen by the sense of sight. However, as for image data obtained by video camera, the user can view only scenery appearing in an image corresponding to the image data.    Patent Document 1: Japanese Patent Laid-Open No. 2001-320587